The goal of most image segmentations is to distinguish or separate an object or objects, often referred to as the “figure” or “foreground”, from the rest of the image. The rest of the image is often referred to as “background” or “ground.” To perform the segmentation process, an image mask or “mask” is generally utilized. Masking figural objects or subjects from an image is an important application and has been a laborious task for decades. Some have attempted to simplify the masking process by utilizing techniques that would assist in distinguishing the background from the foreground, such as photographing foreground objects against a uniform colored background and then selecting any pixels within the image that are that color as background. This approach is still in use by many within the movie and television industries.
Other more laborious approaches include marking up one pixel at a time using commercial software such as Photoshop. In recent years, such commercial applications have implemented approaches to reduce burden on users by power-assist methods. Examples of power-assist methods in Photoshop are: “lasso”; “magnetic-lasso”; “magic wand”; and “quick-select”. Some applications, such as the PhotoShop plug-in Fluid Mask, allow user to select JigCut regions (as defined later). The concept is to allow users to select macro regions instead of micro pixels.
The next level of ease-of-use dawned with the introduction of tri-maps. A tri-map is an image where the figure, ground and other unknown regions are assigned unique labels. A PhotoShop plug-in called Knockout requires the user to provide an accurate tri-map. An application called GrowCut expects user to define a rough region (say, a rectangle) that completely contains the figural subject and then requires user to select samples of figure and ground inside that rough region by means of scribbles. Scribbles are derived from the trajectory of mouse, or equivalent hardware such as marking the figure and the ground samples by scribbling with a pen or brush.
Lazy Snapping (as proposed by, “Lazy snapping”, Y. Li, J. Sun, C. Tang, H. Shum, ACM Transactions on Graphics, special issue: Proceedings of the SIGGRAPH Conference, pp. 303-308, 2004.) requires users to select figure and ground seeds using scribbles, without the need to mark rough region that encompasses the figural object. It approximates the figure region into a polygon and allows the user to adjust the vertices to refine the resulting mask to correct for segmentation error due to low contrast regions.
Soft Scissors (as proposed by, “Soft scissors: an interactive tool for real time high quality matting”, ACM Transactions on Graphics, Proceedings of the SIGGRAPH Conference, vol. 26, issue 3, July 2007.) requires users to roughly trace the contour of boundary and adapts the brush size based on local variation. It uses a restricted form of scribbles in that the scribbles follow the contour, and the figure and ground seeds are selected from outside the scribble region.
As seen from above, several approaches exist for addressing the masking process but with different and difficult variations of user interaction. As should be apparent, there is need for a solution that provides a simple and user-friendly interface for the creation or refinement of an image mask.